Display TensorRT-LLM AgentX server metrics - #747
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
83d2f0b to
4e703c0
Compare
|
Closing because the project scope is now limited to porting active, nondeprecated AMD multi-node configurations to srt-slurm. This PR is outside that scope. |
|
Reopening: this PR predates the recent out-of-scope configuration-porting detour and was closed by mistake during an overly broad cleanup. Its prior state is being restored. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| }; | ||
|
|
||
| const ADAPTERS: readonly ServerMetricsAdapter[] = [dynamoAdapter, genericAdapter]; | ||
| const ADAPTERS: readonly ServerMetricsAdapter[] = [trtllmAdapter, dynamoAdapter, genericAdapter]; |
There was a problem hiding this comment.
TRT-LLM adapter drops Dynamo router role
Medium Severity
The new trtllm adapter is listed before the Dynamo adapter and matches any framework whose name contains trt, including canonical dynamo-trt. Its role mapping handles prefill/backend/decode/aggregated but not Dynamo frontend/router, so those sources are stored as unknown instead of router. Per-source pickers for Dynamo TRT-LLM disagg runs therefore mislabel the frontend endpoint.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 4e703c0. Configure here.
4e703c0 to
1e48786
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
There are 3 total unresolved issues (including 2 from previous reviews).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit c3acc10. Configure here.
1 participant
Summary
Follows SemiAnalysisAI/InferenceX#2625 and NVIDIA/TensorRT-LLM#12545.
Validation
Note
Medium Risk
Bumps stored chart-series semantics and runs a staging DB backfill; changes are localized to metric ETL/adapters with tests, but incorrect TRT-LLM mapping could mislabel disagg sources or chart values.
Overview
Adds TensorRT-LLM support to AgentX server-metric charts and rollups by mapping native
trtllm_*Prometheus series into the same KV, prefix-cache, queue, throughput, and per-source prompt views used for vLLM/SGLang.CHART_SERIES_VERSIONis bumped to 17, including deriving prompt throughput fromtrtllm_prefill_batch_tokenshistogram sums when prompt-token counters are absent, and wiring Dynamo disaggregated prefill/decode via a dedicatedtrtllmserver-metrics adapter (dynamo_component/disaggregation_mode).Aggregate extraction and oversized-blob stream parsing are extended with the same TRT-LLM keys and fallbacks (e.g. direct
trtllm_kv_cache_hit_ratewhen hit/query counters are missing).Staging workflow: after ingest, a new job runs
db:backfill-chart-series --run-idso staged traces get recomputed chart series; staging success now requires that step. The backfill CLI gains--run-idto scope recomputation to traces tied to one GitHub Actions run.Reviewed by Cursor Bugbot for commit 80f9aec. Bugbot is set up for automated code reviews on this repo. Configure here.